High Efficiency Video Coding (HEVC) is a video coding standard currently being developed in Joint Collaborative Team-Video Coding (JCT-VC). JCT-VC is a collaborative project between Moving Picture Experts Group (MPEG) and International Telecommunication Union-Telecommunication Standardization Sector (ITU-T). Currently, an HEVC Model (HM) is defined that includes a number of tools and is considerably more efficient than H.264/Advanced Video Coding (AVC).
A picture coded/decoded according to HEVC is partitioned into one or more slices, where each slice is an independently decodable segment of the picture. This means that if a slice is missing, for instance got lost during transmission, the other slices of that picture can still be decoded correctly. In order to make slices independent, they are self contained and do not depend on each other which imply that no bitstream element of another slice is required for decoding any element of a given slice.
Each slice contains a slice header which independently provides data for the slice to be independently decodable. One example of a data element present in the slice header is the slice address, which is used for the decoder to know the spatial location of the slice. There are many more data elements in the slice header.
HEVC uses reference pictures for inter prediction for decoding. The reference pictures are previously decoded pictures to be used for decoding of a current picture. The decoder has a decoded picture buffer (DPB), which contains previously decoded pictures that are stored by the decoder. The main reason for a previously decoded picture to be stored in the DPB is in order for the decoder to use the picture for inter prediction. The DPB size is limited and the encoder decides what pictures to store; each slice header contains that information. A reference picture in HEVC is a picture in the DPB that is available for reference by being indicated as used for reference. There may also be pictures in the DPB that are marked unused for reference, these pictures are not available for reference and are thereby not reference pictures. The reference pictures in the DPB are either marked as used for short-term reference pictures or used for long-term reference pictures. Short-term pictures are typically kept for a shorter while compared to long-term pictures in the DPB. Another difference between the short term and the long-term pictures is e.g. motion vector scaling, where motion vectors from long-term pictures are not scaled by the POC distance but short-term picture motion vectors are.
Therefore, the HEVC standard includes a picture marking process in which reference pictures can be marked as “used for short-term reference”, “used for long-term reference” and “unused for reference”. If marked as “unused for reference”, a reference picture is turned into a non-reference picture and can not be used for prediction any more.
The reference picture marking process in HEVC is controlled by reference picture sets (RPSs). An RPS is a set of picture identifiers that indicates the pictures that are allowed to be used as reference pictures. The RPS is sent from the encoder to the decoder in each slice header and reference pictures that are indicated in the RPS will be kept in the decoded picture buffer (DPB) which stores the decoded pictures in the decoder. RPS slice header syntax of the long-term picture is shown in table 1 below.
There are various alternatives available that could be used as picture identifier. For instance, the picture identifier could be the decoding order number, the display order number, the output order number or a combination of display order number and an additional identifier or indeed any other information that can be used to unambiguously identify the picture. An examples of such picture identifier include Picture Order Count (POC).
POC is used in HEVC to indicate reference pictures in the RPS and also to define the output order (or display order) of pictures. Each slice contains a POC code word, pic_order_cnt_lsb, that shall be the same for all slices of a picture. pic_order_cnt_lsb is also known as the least significant bits (lsb) of the full POC since is it a fixed-length code word and only the least significant bits of the full POC is signaled. Short-term pictures are indicated in the RPS through a pair of numbers: the POC of the reference picture and a flag: used_by_curr_pic_lx_flag. The decoder knows the POC of the reference pictures in the DPB and can match those against the POC values received in the RPS. The flag used_by_curr_pic_lx_flag indicates whether the reference picture is used for reference for the current picture or not. If the flag indicates that a picture P is not used for reference for the current picture, picture P is still kept in the DPB, but the usage of picture P as a reference picture for the current picture is prohibited. It may well be the case that picture P is used as a reference picture for a future picture however. Further, the POCs indicated in the RPS can be represented by delta POCs relative to the POC of the current picture, wherein the POC of the current picture is known. To determine the POC, the decoder has to decode the syntax elements with the delta POC and add their values with the POC of the current picture.
Long-term pictures are generally indicated in the RPS by the least significant bit (lsb) part of the POC value of respective long-term picture. However, the HEVC standard has an optional syntax element, delta_poc_msb_cycle_lt_minus1, which provides an alternative way of referencing long-term pictures. The long-term picture part of the HEVC slice header syntax is shown in table 1.
TABLE 1RPS slice header syntax for long-term picturesslice_header( ) {Descriptor ... if( long_term_ref_pics_present_flag ) { num_long_term_picsue(v) for( i = 0; i < num_long_term_pics; i++ ) {  delta_poc_lsb_lt[ i ]ue(v)  delta_poc_msb_present_flag[ i ]u(1)  if( delta_poc_msb_present_flag[ i ] )  delta_poc_msb_cycle_lt_minus1[ i ]ue(v)  used_by_curr_pic_lt_flag[ i ]u(1) } } ...}
If delta_poc_msb_present_flag is equal to 0, the long-term picture is indicated by the lsb part of its POC. If delta_poc_msb_present_flag is equal to 1, the long-term picture is indicated by the full POC. delta_poc_msb_present_flag equal to 1 allows for two long-term pictures to share the same POC lsb. The syntax element, delta_poc_msb_cycle_lt_minus1 will be present when delta_poc_msb_present_flag is equal to 1. The existing HEVC standard mandates that delta_poc_msb_present_flag shall be equal to 1 for every long-term picture in the RPS that has identical POC lsb to another long-term picture in the RPS.